7 research outputs found
nerf2nerf: Pairwise Registration of Neural Radiance Fields
We introduce a technique for pairwise registration of neural fields that
extends classical optimization-based local registration (i.e. ICP) to operate
on Neural Radiance Fields (NeRF) -- neural 3D scene representations trained
from collections of calibrated images. NeRF does not decompose illumination and
color, so to make registration invariant to illumination, we introduce the
concept of a ''surface field'' -- a field distilled from a pre-trained NeRF
model that measures the likelihood of a point being on the surface of an
object. We then cast nerf2nerf registration as a robust optimization that
iteratively seeks a rigid transformation that aligns the surface fields of the
two scenes. We evaluate the effectiveness of our technique by introducing a
dataset of pre-trained NeRF scenes -- our synthetic scenes enable quantitative
evaluations and comparisons to classical registration techniques, while our
real scenes demonstrate the validity of our technique in real-world scenarios.
Additional results available at: https://nerf2nerf.github.i
Attention Beats Concatenation for Conditioning Neural Fields
Neural fields model signals by mapping coordinate inputs to sampled values.
They are becoming an increasingly important backbone architecture across many
fields from vision and graphics to biology and astronomy. In this paper, we
explore the differences between common conditioning mechanisms within these
networks, an essential ingredient in shifting neural fields from memorization
of signals to generalization, where the set of signals lying on a manifold is
modelled jointly. In particular, we are interested in the scaling behaviour of
these mechanisms to increasingly high-dimensional conditioning variables. As we
show in our experiments, high-dimensional conditioning is key to modelling
complex data distributions, thus it is important to determine what architecture
choices best enable this when working on such problems. To this end, we run
experiments modelling 2D, 3D, and 4D signals with neural fields, employing
concatenation, hyper-network, and attention-based conditioning strategies -- a
necessary but laborious effort that has not been performed in the literature.
We find that attention-based conditioning outperforms other approaches in a
variety of settings
A Novel Tool for the Assessment of Pain: Validation in Low Back Pain
Joachim Scholz and colleagues develop and validate an assessment tool that distinguishes between radicular and axial low back pain
VIPER: Volume Invariant Position-based Elastic Rods
International audienceWe extend the formulation of position-based rods to include elastic volumetric deformations. We achieve this by introducing an additional degree of freedom per vertex-isotropic scale (and its velocity). Including scale enriches the space of possible deformations, allowing the simulation of volumetric effects, such as a reduction in cross-sectional area when a rod is stretched. We rigorously derive the continuous formulation of its elastic energy potentials, and hence its associated position-based dynamics (PBD) updates to realize this model, enabling the simulation of up to 26000 DOFs at 140 Hz in our GPU implementation. We further show how rods can provide a compact alternative to tetrahedral meshes for the representation of complex muscle deformations, as well as providing a convenient representation for collision detection. This is achieved by modeling a muscle as a bundle of rods, for which we also introduce a technique to automatically convert a muscle surface mesh into a rods-bundle. Finally, we show how rods and/or bundles can be skinned to a surface mesh to drive its deformation, resulting in an alternative to cages for real-time volumetric deformation. The source code of our physics engine will be openly available